Restore lightrag-hku 1.4.x compatibility and drop hardcoded 1536-dim embeddings - #8
Open
mmavka wants to merge 1 commit into
Open
Restore lightrag-hku 1.4.x compatibility and drop hardcoded 1536-dim embeddings#8mmavka wants to merge 1 commit into
mmavka wants to merge 1 commit into
Conversation
…embeddings Three small fixes that together let knowledge-mcp work against the latest lightrag-hku (1.4.16) plus arbitrary embedding dimensions. 1. Drop llm_model_max_token_size from the LightRAG constructor call in rag.py. The kwarg was removed upstream in 1.4.x, so KB creation was failing with TypeError before this. 2. Filter ids, model_func, chunk_top_k, and enable_rerank out of QueryParam kwargs before construction. Those fields were removed from QueryParam upstream but still ship in the default per-KB config template, so every query was crashing. 3. Replace the openai_embed wrapper in openai_embedding_func with a direct AsyncOpenAI client call. openai_embed is decorated with wrap_embedding_func_with_attrs(embedding_dim=1536), which silently pinned the embedding dimension to OpenAI text-embedding-3-* and broke any non-1536-dim endpoint (Ollama bge-m3, mxbai-embed-large, etc.). Tested locally against lightrag-hku 1.4.16, OpenRouter as the LLM endpoint, and Ollama bge-m3 (1024-dim) for embeddings. KB create, add_text, and all five query MCP tools work end-to-end after the patches; before, each issue blocks at a different stage.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Trying to run knowledge-mcp 0.4.2 against the latest
lightrag-hku(1.4.16) with a local Ollama setup hit three errors in a row. This PR fixes all of them.What's broken
1.
llm_model_max_token_sizeremoved in lightrag-hku 1.4.xLightRAG.__init__()no longer acceptsllm_model_max_token_size. Creating a KB fails with:Dropped the kwarg from the constructor call in
rag.py. The behaviour it used to control is now handled elsewhere in lightrag-hku and doesn't need to be set explicitly here.2. Several
QueryParamfields removed in lightrag-hku 1.4.xThe per-KB config template populates
QueryParamwithids,model_func,chunk_top_k, andenable_rerank. None of those exist in the currentQueryParamdataclass, so any query crashes with:(or one of the other three, depending on dict iteration order).
Filtered those four keys out of
final_query_paramsbefore theQueryParamconstruction. The existingdescriptionfilter right above is doing the same dance, so I slotted the rest in next to it.This is the conservative fix: the per-KB config can keep emitting those fields without breaking, and if any come back upstream we don't have to touch anything else.
3.
openai_embedis hardcoded to 1536 dimsopenai_embedding_funcinopenai_func.pydelegates tolightrag.llm.openai.openai_embed, which is wrapped with@wrap_embedding_func_with_attrs(embedding_dim=1536)(i.e. OpenAI text-embedding-3-* dimensions baked in via decorator).Fine for OpenAI, but anyone pointing the embedding endpoint at an OpenAI-compatible server with a different model (Ollama
bge-m3at 1024 dims,mxbai-embed-largealso 1024, etc.) gets:embedding_dimis already configurable inconfig.yaml(lightrag.embedding.embedding_dim), but never honoured because the decorator onopenai_embedoverrides it.Rewrote the inner function to call the OpenAI-compatible endpoint directly with
AsyncOpenAIinstead of going throughopenai_embed. Six lines, respects whatever dimension the endpoint actually returns. TheEmbeddingFuncwrapper just below (lines 90–93) still readsembedding_dimfrom config, so dimensions are validated correctly downstream; they're just no longer pinned to 1536 in the inner function.How tested
Local setup:
bge-m3(1024-dim) athttp://localhost:11434/v1After the patches:
create <kb>andadd_text <kb> <file>succeed; entity/relation extraction runs cleanly through OpenRouteranswer,query_hybrid,query_local,query_global,retrieve) all return synthesised answers with citationsBefore the patches each of the three issues blocks at a different stage (create / first query / first embedding call).
Notes
pyproject.toml. Pinning to a known-good range would be more solid long-term, but it's outside what I needed today.RuntimeError: no running event loopat shell exit. Unrelated, doesn't affect ingest or query results. Can look at it separately if useful.Let me know if you'd rather split this into separate commits (compat fixes vs the embed change). They're independent, and I can rebase.